The present invention relates generally to optical networks, and more particularly, to a switching procedure for multiple-input multiple-output-orthogonal-frequency-division-multiplexing (MIMO-OFDM) based flexible rate Intra-data center network.
The following prior documents referenced in the background discussion, not material to patentability of the claimed invention, nevertheless provide additional information.
General Survey of DCN Architectures:                [Kachris] C. Kachris and I. Tomkos, “A Survey on Optical Interconnects for Data Centers”, IEEE Communications Surveys and Tutorials, Vol. 14, Iss. 4, pp. 1021-1036, 2012        
Inventor's Prior Works on MIMO OFDM DCN:                [Ji1] P. N. Ji, T. Wang, and Y. Aono, “MIMO-OFDM-based flexible rate intra-data center network”, NECLA IR 11101;        [Ji2] P. N. Ji, T. Wang, et al., “Demonstration of High-Speed MIMO OFDM Flexible Bandwidth Data Center Network”, Proceedings of ECOC 2012, paper Th.2.B.1, 2012;        [Ji3] P. N. Ji, D. Qian, et al., “Design and Evaluation of a Flexible-Bandwidth OFDM-Based Intra Data Center Interconnect”, IEEE Journal of Selected Topics in Quantum Electronics (in press)        
Background of OBS:                [Qiao] C. Qiao and M. Yoo, “Optical Burst Switching (OBS)—A New Paradigm for an Optical Internet”, Journal of High Speed Networks, Vol. 8, Iss. 1, pp. 69-84, 1999        [Chen] Y. Chen, C. Qiao & X. Yu, “Optical Burst Switching (OBS): A New Area in Optical Networking Research”, IEEE Journal of Networking, Vol. 8, Iss. 3, pp. 16-23, 2004        
Introduction of Intune's OPST Technology:                [Dunne] J. Dunne, “Design Principles for Optical Packet Switch and Transport (OPST) Networks”, Intune white paper, 2010        
Introduction of Software-Defined Optical Network:                [Ji4] P. N. Ji, “Software Defined Optical Network”, Proceedings of 11th International Conference on Optical Communications and Networks (ICOCN 2012), paper THU-07, 2012        
As the global Internet traffic growing exponentially, the data centers, which host many Internet application servers, are also facing rapid increase in bandwidth demands. In recent years, large-scale data centers continued to be built out. Due to emerging applications such as cloud computing and “Big data” processing, next generation data centers need to achieve low latency, high throughput, high flexibility, high re-source efficiency, low power consumption, and low cost. Furthermore, as more and more processing cores are integrated into a single chip, the communication requirements between racks in the data centers will keep increasing significantly. By integrating hundreds of cores into the same chip (e.g. Single-chip Cloud Computer) we can achieve higher processing power in the data center racks. However these cores require a fast and low-latency interconnection scheme to communicate with the storage system and the other servers inside or outside of the rack. This communication network between racks is referred to as the intra-data center network, as oppose to the inter-data center network for long distance communication between data centers at different geographical locations. In the following, we focus on the intra-data center network and will thus simply refer it as the data center network (DCN).
Optical technology has been adopted in DCN due to its high bandwidth capacity. However, it is mainly used for point-to-point link, while the DCN interconnect is still based on electrical switching fabric, which has high power consumption and limited bandwidth capacity. Currently, the power consumption of the data center networks account for 23% of the total IT power consumption. However, due to the anticipated high communication requirements in the future, it is estimated that the DCN will account for much higher percentages of the overall power consumption. Therefore it is expected that data center network may evolve to all-optical networks, similarly to the telecommunication networks that have been evolved from opaque to transparent networks using all-optical switches.
In recent years, several hybrid optical/electrical (O/E) or all-optical interconnect schemes for DCN have been proposed [Kachris]. Many of them rely on large scale fiber cross-connect (FXC) or multiple wavelength-selective switches (WSS), which are costly and have slow switching speed (at millisecond level). Having a large scale FXC also present an undesirable single source-of-failure. A recent work used silicon electro-optic mirroring WSS and semiconductor optical amplifier-based switch to achieve nanosecond scale switching, making all-optical packet level routing possible. However the key components are not commercially available and have low scalability. Other architectures use tunable wavelength converters (TWC). They are also costly and do not allow bandwidth resource sharing among the connections. Some of them also require electrical or optical buffers.
In most of these all-optical or hybrid O/E DCN architecture designs, the switching is performed at the circuit level because the optical switch hardware is not fast enough to process IP packets (which is the main type of traffic in data centers). This is called optical circuit switching (OCS). This limitation makes the optical DCN solution less efficient and thus less attractive. Some hybrid O-E architectures use the optical switching portion only for large volume, slow varying traffics, and still rely on the electrical portion for regular IP traffics. Having both types of switching hardware in a DCN requires high equipment cost and cannot solve the high power consumption problem in electrical switching DCN.
Recently we proposed and experimentally demonstrated a novel all-optical DCN architecture that combines a passive cyclic arrayed waveguide grating (CAWG) core router with orthogonal frequency division multiplexing (OFDM) modulation and parallel signal detection (PSD) technologies [Ji1, Ji2, Ji3]. FIG. 1 is the schematic of the DCN architecture. The architecture achieves fast switching (nanosecond speed capable) with low and uniform latency (single hop), low power consumption, and multiple-input multiple-output (MIMO) switching capability, while allowing fine granularity bandwidth sharing (at Mb/s level) and having low cost (does not require any FXC, WSS, or TWC). We also proposed and analyzed various subcarrier allocation algorithms to efficiently use the bandwidth resource and optimize the throughput in the DCN.
However, the MIMO-OFDM DCN research so far is on physical layer hardware implementation, as well as subcarrier allocation algorithm. There is no control system to utilize them in practical DCN application. Furthermore, despite the high switching speed capability, the work so far is based on circuit switching only. In other words, there is no solution to enable packet switching in this DCN architecture.
Optical packet switching (OPS) has been studied for many years to enable packet switching function in the optical domain, because it will provide better flexibility, efficient resource utilization, potential functionality and finer switching granularity. Some OPS prototype systems have been built and demonstrated. However, OPS is not efficient because in OPS the data payload must wait in the optical buffers before it can be forwarded to the next node since the packet header needs to be processed either all-optically or electronically after an optical-to-electrical (O-E) conversion at each intermediate node. And there are many major challenges for it to be solved before OPS can be practical enough for actual application. For example, there is no optical equivalent of RAM and logic devices for optical signal processing (such as optical packet header processing). And even if the header is processed electronically, there is no flexible optical buffer with practical buffering capability currently available (a less flexible alternative, namely the fiber delay line, is currently used in the OPS research testbeds), and the optical switching hardware at packet speed level is still very expensive and large.
A good compromise between OCS and OPS is optical burst switching (OBS) [Qiao, Chen]. In an OBS network, various types of client data (including asynchronous traffics such as Ethernet, and synchronous traffics such as SONET, SDH, OTN, Fiber Channel, FICON . . . ) are aggregated at the ingress (an edge node) and transmitted as data bursts (FIG. 2(a)) which later will be disassembled at the egress node (FIG. 2(b)). During burst assembly/disassembly, the client data is buffered at the edge where electronic RAM is cheap and abundant. Even though the client data go through burst assembly/disassembly only at the edge of an OBS network, nevertheless, statistical multiplexing at the burst level can still be achieved in the core of the OBS network.
Another feature of OBS is that data and control signals are transmitted separately on different channels or wavelengths, as illustrated in FIG. 3. For each data burst, a control packet containing the usual “header” information (such as the burst length information) is transmitted on a dedicated control channel. Since a control packet is significantly smaller than a burst, one control channel is sufficient to carry control packets associated with multiple (e.g., hundreds of) data channels. A control packet goes through OEO conversion at each intermediate OBS node and is processed electronically to configure the underlying switching fabric. There is an offset time between a control packet and the corresponding data burst to compensate for the processing/configuration delay. For multi-hop network, the offset time at each hop is different, as illustrated in FIG. 3. If the offset time is large enough, the data burst will be switched all-optically and in a “cut-through” manner, i.e., without being delayed at any intermediate node (core). In this way, no optical RAM or fiber delay lines is necessary at any intermediate node. This is an advantage over OPS. Furthermore, it allows a lower control overhead per bit than that in OPS. In OBS, costly OEO (optical to electrical to optical) conversions are only required on a few control channels instead of a large number of data channels. Compared to OCS, the burst-level granularity leads to a statistical multiplexing gain and thus more efficient use of the bandwidth, which is absent in OCS.
Therefore OBS presents a good solution for DCN because it combines the advantages of optical switching (low power consumption, high capacity) and the advantage of sub-wavelength switching (statistical multiplexing gain, better bandwidth utilization) in a practical way.
So far, there is only one carrier grade OBS architecture and product in the world. This is developed by Intune Networks. The technology is called the Optical Packet Switch and Transport (OPST) and is based on their fast tunable optical transmitters [Dunne]. Despite what the name (which includes “optical packet switch”) suggests, the OPST is actually a type of OBS technology. In fact, it was claimed to be “the world's first carrier grade OBS architecture”. In such architecture (FIG. 4), every node is attached to the OPST fabric (ring) through a fast tunable laser (with nanoseconds tuning speed) and a burst mode receiver (BMR) under that port's local control. Each receiver node is assigned a unique wavelength in which every other node can transmit to this node by tuning the transmitter in real-time. At each BMR, the ingress packets are queued by class of service per wavelength. Each port can send and receive frames to and from all other ports, providing a fully distributed switching capability with no intervening OEO between source and destination.
Intune's OPST architecture is most suitable for shorter distance networks (up to a few hundreds of kilometers). Although it is advertised mainly for transport networks, it could be also used to replace the core network of the data centers [Kachris].
There are several limitations to this OPST architecture. For example, due to its ring topology, it is difficult to inset new nodes without interrupting the existing traffics, because even though each node has its own dedicated wavelength, all wavelengths are transmitted within the same fiber using wavelength division multiplexing (WDM). Therefore in-service upgrade cannot be achieved. Secondly, at the receiver of each node, a passive optical coupler is used to combine all the signals from all other nodes. If the coupler has large port count to accommodate all existing and potential future nodes, the optical loss will be very large. On the other hand, if the port number of the coupler is limited to a certain quantity, the number of nodes that can simultaneously transmit to the particular destination node will be limited too. Therefore this architecture is not very scalable. So far the product can support only up to 16 nodes in a ring, which is not sufficient for regular data centers. Thirdly, the ring topology also leads to the fact that different source-destination pairs go through different numbers of hops (one hop is between two adjacent node), such as the yellow traffic vs. the red traffic in FIG. 4, making the propagation delay uneven, and thus the latency is also not uniform.
Accordingly, there is a need for a switching procedure for MIMO-OFDM based flexible rate intra-data center network that overcomes the shortcomings of prior efforts.